Integrating Time Alignment and Neural Networks for High Performance Continuous Speech Recognition

Successful application of existing connectionist methods to continuous speech recognition requires the use or time-alignment procedures. These procedures. usually based on dynamic programming, provide means for supervising the training of neural networks. This paper describes two systems in which neural network classifiers are merged with dynamic programming (DP) time alignment methods to produce high performance continuous speech recognizers. One system uses the Connectionist Viterbi Training (CVT) procedure, in which a neural network with frame-level outputs is trained using guidance from a time alignment procedure. The other system uses Multi-State Time Delay Neural Networks (MS-TDNNs). in which embedded DP time alignment allows network training with only word-level external supervision. CVT has been described previously [l] ; only changes lo the system and new results on the TI Digits task are reported here. The newest CVT results on the TI Digits are 99.1% word accuracy and 98.0% string accuracy. MSTDNNs, introduced in this paper, are described in more detail here, with attcntion focused on their basic architecture, the training procedure, and results of applying MS-TDNNs to continuous speakerdependent alphabet recognition: on two speakers, word accuracy is respectively 97.5% and 89.7%.